Multi-mic sound collector and system and method for sound localization

ABSTRACT

A sound localization system includes a multi-mic sound collector and a computing device. The multi-mic sound collector includes a carrier and a plurality of sound receivers removably attached to the sound receivers. The computing device includes a data communicator, a synchronizer, and a processor. The data communicator receives preprocessed audio data from the multi-mic sound collector. The synchronizer synchronizes the preprocessed audio data. The processor analyzes the synchronized audio data to identify and localize a target audio feature. A sound localization method of the sound localization system is also provided. The present invention facilitates monitoring of the functioning or physiological signs of a monitored subject and allows early detection and diagnosis of abnormalities or diseases.

CROSS-REFERENCE TO RELATED APPLICATION

The present disclosure claims the benefit of U.S. ProvisionalApplication No. 64/420,573, filed on Nov. 11, 2016.

FIELD OF THE INVENTION

The present disclosure relates generally to a system and method forsound localization, and more particularly to a multi-mic sound collectorand a system and method for precise localization of abnormal sounds.

BACKGROUND OF THE INVENTION

Functioning of a subject can be assessed by analysis of the soundscoming from inside of the subject. For example, abnormal sounds made byan operating machine could be an indicator for malfunction of thecomponents of the machine. Similar in medical and healthcare fields,auscultation of the sounds caused by movements of various organs of thebody has been a common tool for diagnosis of the health conditions of alive subject.

Conventional sound analysis samples sounds at one spot at a time andrequires professionals that have been specifically trained to identifyabnormal sounds from regular ones and environmental noise. However,single spot detection often results in limited and sometimes misleadinginformation. For example, as pathological breathing sound feature variesby symptoms and spots of auscultation, conventional single spotauscultation has not been a useful tool for diagnosis of asthma.

Furthermore, although the emergence of electronic sound analyzers hasfacilitated the sound analysis process, precise localization of thesource of abnormal sounds has remained a challenge.

BRIEF SUMMARY OF THE INVENTION

An embodiment of the present invention provides a sound localizationsystem. The sound localization system includes a multi-mic soundcollector and a computing device. The multi-mic sound collector includesa plurality of sound receivers for collecting sound data of a subject.The computing device is in communication with the multi-mic soundcollector, and includes a data communicator for receiving preprocessedaudio data from the multi-mic sound collector, a synchronizerelectrically connected to the data communicator for synchronizing thepreprocessed audio data, and a processor electrically connected to thesynchronizer for analyzing the synchronized audio data to identify andlocalize a target audio feature.

Preferably, the multi-mic sound collector includes a carrier and aplurality of sound collecting modules removably attached to the carrier.Each of the sound collecting modules includes one of the plurality ofsound receivers, a convertor electrically connected to the soundreceiver for converting the sound data into digital audio data; and amicrocontroller (MCU) electrically connected to the convertor forcontrolling sound collection by the sound receiver and preprocessing thedigital audio data.

Preferably, the multi-mic sound collector includes a carrier, theplurality of sound receivers removably attached to the carrier, and apreprocessing device. The preprocessing device includes a convertorelectrically connected to the sound receivers for converting the sounddata into digital audio data; and a MCU electrically connected to theconvertor for controlling sound collection by the sound receivers andpreprocessing the digital audio data.

Preferably, the processor of the computing device includes a featureextractor for identifying and extracting preliminary audio features fromthe synchronized audio data, a classifier for separating and classifyingthe preliminary audio features to obtain the target audio feature, and asignal localizer for analyzing the target audio feature to obtainlocational information of a source of the target audio feature.

Preferably, the processor further includes a data analyzer for comparingthe obtained target audio feature and the location of the source of thetarget audio feature with data stored in the computing device to obtaina diagnostic result.

Preferably, the computing device is further in communication with aserver for data analysis and storage.

Preferably, the sound receivers are a plurality of stethoscopicchestpieces for auscultating the subject.

Preferably, the amount of the sound receivers are at least three.

Preferably, at least a portion of the sound receivers are arrangedrectangularly over a chest of the subject.

Preferably, at least a portion of the sound receivers are arrangedrectangularly over a heart of the subject.

Preferably, at least a portion of the sound receivers are arrangedtriangularly over a heart of the subject.

Preferably, at least a portion of the sound receivers are arranged atleft and right costophrenic angles at a posterior chest of the subject.

An embodiment of the present invention provides a sound localizationmethod for the aforementioned sound localization system. The methodincludes the steps of: acquiring sound data of the subject; identifyinga target audio feature from the sound data; and analyzing the targetaudio feature to obtain locational information of a source of the targetaudio feature.

Preferably, the step of identifying a target audio feature from thesound data includes the steps of: preprocessing the sound data;extracting preliminary audio features; and separating and classifyingthe preliminary audio features to obtain the target audio feature.

Preferably, the step of extracting preliminary audio features isperformed according to a voice activity detector (VAD) algorithm, aMel-frequency cepstral coefficient (MFCC) algorithm, and a K-meansalgorithm.

Preferably, the step of separating and classifying the extractedpreliminary audio features includes the steps of: separating noise fromthe preliminary audio features; classifying normal and abnormal audiofeatures; and separating undesired abnormal audio features to obtain thetarget audio feature.

Preferably, the step of separating and classifying the extractedpreliminary audio features is performed according to a K-nearestneighbor (KNN) algorithm, a Gaussian mixture model (GMM) algorithm, asupport vector machine (SVM) algorithm, or a deep neural network (DNN)algorithm.

Preferably, the step of analyzing the target audio feature includes astep of: performing direction of arrival (DOA) estimations on the targetaudio feature to obtain the locational information of the source of thetarget audio feature.

Preferably, the sound localization method further includes a step of:comparing the target audio feature and the locational information of thesource of the target audio feature with stored data to obtain adiagnostic result.

Preferably, the sound localization method further includes a step of:visualizing the locational information of the source of the target audiofeature over a multi-dimensional image of the subject.

In sum, the present invention according to the preferred embodimentscouples a multi-mic sound collector with sound analysis and spatialanalysis to identify abnormal or pathological sounds coming from amonitored subject and obtain detailed locational information of thesource of the abnormal sounds. The present invention facilitatesmonitoring of the functioning or physiological signs of the subject andallows early detection and diagnosis of abnormalities or diseases.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate one or more embodiments of thepresent invention and, together with the written description, explainthe principles of the present invention. Wherever possible, the samereference numbers are used throughout the drawings to refer to the sameor like elements of an embodiment.

FIG. 1 is a schematic illustration of a sound localization system inaccordance with an embodiment of the present invention;

FIG. 2 is a block diagram of a sound localization system in accordancewith an embodiment of the present invention;

FIG. 3 is a schematic illustration of a sound localization system inaccordance with another embodiment of the present invention;

FIGS. 4A-4B are schematic illustrations of carriers of the multi-micsound collector of the sound localization system in accordance withvarious embodiments of the invention;

FIGS. 5A-5C are schematic illustrations of the arrangement of the soundreceivers of the multi-mic sound collector in accordance with variousembodiments of the invention;

FIG. 6 is a block diagram of a processor of a computing device of asound localization system in accordance with an embodiment of thepresent invention;

FIG. 7 is a flow diagram showing the steps of a sound localizationmethod in accordance with an embodiment of the present invention;

FIGS. 8A-8C are schematic illustrations of the results of spatialanalysis in accordance with various embodiments of the presentinvention;

FIGS. 9A-9B are schematic illustrations of the user interface of anapplication in the computing device of the sound localization system inaccordance with an embodiment of the present invention;

FIG. 10 is a flow diagram showing the steps of breathing soundmonitoring and analysis in accordance with an exemplary embodiment ofthe present invention; and

FIG. 11 is a flow diagram showing the steps of breathing soundmonitoring and analysis in accordance with an exemplary embodiment ofthe present invention.

In accordance with common practice, the various described features arenot drawn to scale and are drawn to emphasize features relevant to thepresent disclosure. Like reference characters denote like elementsthroughout the figures and text.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will now be described more fully hereinafter withreference to the accompanying drawings illustrating various exemplaryembodiments of the invention. The present invention may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein. Rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the disclosure to those skilled in the art.Like reference numerals refer to like elements throughout.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” or “includes” and/or “including” or “has” and/or“having” when used herein, specify the presence of stated features,regions, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components, and/orgroups thereof.

It will be understood that the term “and/or” includes any and allcombinations of one or more of the associated listed items. It will alsobe understood that, although the terms first, second, third etc. may beused herein to describe various elements, components, regions, partsand/or sections, these elements, components, regions, parts and/orsections should not be limited by these terms. These terms are only usedto distinguish one element, component, region, part or section fromanother element, component, region, layer or section. Thus, a firstelement, component, region, part or section discussed below could betermed a second element, component, region, layer or section withoutdeparting from the teachings of the present disclosure.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this disclosure belongs. It willbe further understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and thepresent disclosure, and will not be interpreted in an idealized oroverly formal sense unless expressly so defined herein.

The description will be made as to the embodiments of the presentdisclosure in conjunction with the accompanying drawings in FIGS. 1 to11. Reference will be made to the drawing figures to describe thepresent disclosure in detail, wherein depicted elements are notnecessarily shown to scale and wherein like or similar elements aredesignated by same or similar reference numeral through the severalviews and same or similar terminology.

Multi-mic sound collector and system and method for sound localizationin accordance with various embodiments of the present invention areuseful in medical and healthcare environments, such as hospital,healthcare center, etc. Exemplary embodiments may be directed toauscultating live subjects, such as humans, animals, livestock, or othertypes of living beings. Although examples described herein relate toauscultation over certain areas of a subject's body, for example a chestarea, precordia area, abdomen area, extremity area, head area, neckarea, or constituent thereof (for example, lung, gastrointestinalsystem, aorta, tricuspid, brachial artery, femoral artery, trachea,jugular vein, temporal region, mastoid region, etc.), it is not solimited. Those skilled in the art will readily understand thatauscultation over other portions of a subject's body may also beadvantageously used, depending on the desired information andcircumstances. It is therefore appreciated that the disclosed technologymay be suitably configured to auscultate over other areas of the subjectcorresponding to one or more different portion of the body of thesubject. Further, some embodiments of the present invention may beoptionally configured to obtain a subject's biographical or otheridentifiable information and associate the information to theaforementioned auscultation performed by a medical or healthcareprovider.

Exemplary embodiments of the present invention may also be directed tocollection, analysis and localization of sounds generated by non-livingobjects, such as vehicles, machines, pipelines and the like. Otherexemplary embodiments of the present invention may directed tocollection and analysis of sounds in a defined space and localization ofthe sound source within the space.

Referring now to FIG. 1. A sound localization system according to anembodiment of the present invention includes a multi-mic sound collector100 and a computing device 200. The multi-mic sound collector 100 isconfigured to collect sounds coming from a subject 10 to be monitoredand convert the sounds into audio signals. The computing device 200 isin communication with the multi-mic sound collector 100 and isconfigured to receive the acoustic signals transmitted from themulti-mic sound collector 100. The computing device 200 may furthercommunicate with a server 300, such as a cloud server.

Referring now to FIG. 2. The multi-mic sound collector 100 includes aplurality of sound collecting modules 101 and a carrier 102. Each of thesound collecting modules 101 includes a sound receiver 110, a convertor120, a microcontroller (MCU) 130, and a data transmitter 140. The soundcollecting modules 101 may be suitably placed over an object or adefined space, and the sound receiver 110 may be a microphone configuredto collect sounds coming from the object or within the defined space.The sound receiver 110 may also be a stethoscopic chestpiece that aresuitably placed over the body of the subject 10 to collect sounds cominginternal organs of the subject 10, as exemplarily illustrated in FIG. 1.In an alternative embodiment as illustrated in FIG. 3, the multi-micsound collector 100 may include a plurality of the sound receivers 110,a preprocessing device 160, and the carrier 102. The convertor 120, themicrocontroller (MCU) 130, and the data transmitter 140 are integratedinto the preprocessing device 160.

It is to be understood that the sound receivers 110 are not limited tothe aforementioned microphones or stethoscopic chestpieces, but may bemicro-electromechanical (MEMS) microphones, condenser microphones,electret condenser microphones, capacitive microphones, piezoelectricmicrophones, or any sound receivers that can or can be configured tocollect audible sounds and/or inaudible sounds, such as ultrasonic andhypersonic sounds, or any sensors that can or can be configured todetect mechanical vibration in various frequencies and to convert thevibration into electrical signals. In addition, the sound receivers 110of the sound collectors 101 may be identical, or be a combination ofdifferent types of sound receivers. For higher precision in soundlocalization, the multi-mic sound collector 100 preferably includesthree or more sound receivers 110 that are not all arranged on a sameplane.

Further, the sound receivers 110 may have frequency response suitablefor medical use. For example, power attenuation of the sound receivers110 may be less than 12 dB in the frequency range of 100-500 Hz and lessthan 20 dB in the frequency range of 500-1000 Hz. Frequency response ofthe sound receivers 110 may be adjusted for different applications, suchas detection of heart sounds or breath sounds. In general, frequencyresponse of the sound receivers 110 may be optimized to 20-800 Hz forheart sound detection, 20-100 Hz for detection of the first to fourthheart sounds, 20-00 Hz or 100-600 Hz for detection of heart murmurs.Similarly, frequency response of the sound receivers 110 may beoptimized to 200-600 Hz for detection of vesicular breath sounds, and to20-100 Hz for detection of bronchial breath sounds.

The sound receivers 110 are removably attached to the carrier 102 forcontinuous monitoring of a subject. As exemplarily illustrated in FIG.4A, the carrier 102 may be, but is not limited to, a vest, shirt,jacket, suspenders, overall, shortall, sleeve, waistband, or other typesof outfit. Alternatively, as exemplified in FIG. 4B, the carrier 102 maybe a common consumer product, such as a pillow, bath towel, sleep bag,backpack, or car seat. It is to be understood that the carrier 102 mayhave a variety of designs for specific applications or user populations;for example, the carrier may have animal-themed designs for youngchildren or may be integrated with other technical features for morecomfort, convenience, or diversified functions.

The amount and arrangement of the sound receivers 110 on the carrier 102may vary according to different applications. For example, to monitorheart sound and bronchial breath sound of an adult patient, sixstethoscopic chestpieces, as illustrated by dark full circles in FIG.5A, are preferably arranged rectangularly over the heart region at theanterior chest, with each of the chestpieces being spaced apart from theadjacent ones by a distance (for example, 5 cm). Meanwhile, to monitorsound of the entire lung field of the adult patient, nine stethoscopicchestpieces, as illustrated by dark full circles in FIG. 5B, arepreferably arranged rectangularly at the posterior chest, with each ofthe chestpieces being spaced apart from the adjacent ones by a distance(for example, 10 cm). In contrast, as illustrated in FIG. 5C, to monitora young child, three stethoscopic chestpieces are preferably arrangedtriangularly at a higher region at the anterior chest, with each of thechestpieces being spaced apart from the adjacent ones by a distance (forexample, 5 cm) to record sounds of the heart and upper part of the lung;meanwhile, two stethoscopic chestpieces, as illustrated by patternedcircles in FIG. 5C, are arranged at the left and right costophrenicangles at the posterior chest and spaced apart by a distance (forexample, 10 cm) to record sounds of the lower part of the lung of theyoung child. It is to be understood that the preferred embodimentsillustrated in FIGS. 5A-5C are merely exemplary, and that the presentinvention does not limit the amount and arrangement of the soundreceivers to those exemplified in the drawings.

Referring again to FIG. 2. The convertor 120 of the multi-mic soundcollector 100 is electrically connected to the plurality of soundreceivers and is configured to convert and encode the acquiredunprocessed sound data from analog audio data into digital audio data.The MCU 130 is electrically connected to the convertor 120 and isconfigured to control sound collection of the sound receiver 110 andpreprocess, such as down sampling, positioning, tagging, augmenting,filtering, and time stamping, the audio data received from the convertor120. The data transmitter 140 is electrically connected to the MCU 130and is configured to transmit preprocessed audio data to the computingdevice 200. The multi-mic sound collector 100 may further include apower 150 to provide and store electricity for the multi-mic soundcollector 100. The power 150 may be a rechargeable or non-rechargeablebattery. In the case of rechargeable battery, the multi-mic soundcollector 100 may further include a charging port (not shown) forcharging the rechargeable battery. Alternatively, the multi-mic soundcollector 100 may further include a wireless charging circuit to becoupled to a wireless charging dock.

The computing device 200 includes a data communicator 210, asynchronizer 220, a processor 230, a display 240, a memory 250. The datacommunicator 210 is in communication with the data transmitter 140 ofthe multi-mic sound collector 100 and is configured to receivepreprocessed audio data from the multi-mic sound collector 100. Thesynchronizer 220 is electrically connected to the data communicator 210and is configured to synchronize the received preprocessed audio data.The processor 230 is electrically connected to the synchronizer 220 andis configured to analyze the synchronized data to identify and localizeabnormal sounds. The processor 230 is further electrically connected tothe display 240 for data presentation and visualization, the datacommunicator 210 for data communication with other device(s), and thememory 250 for data storage. The display 240 may be a LCD screen, touchpanel, OLED, CRT, projector, or other types of display components. Thememory 260 may be volatile memory or non-volatile memory, such as RAM,ROM, EEPROM, flash memory, optical storage, magnetic disk storage orother magnetic storage devices. The data communicator 210 may furthercommunicate with the server 300 for more data analysis and storage. Itis to be understood that the embodiments of the present inventionprovided herein do not limit the hardware in which data processing andanalyses take place; that is, some of the functions performed by theprocessor of the computing device 200 may be executed by the server 300,and vice versa. The computing device 200 may further include a power 260to provide and store electricity for the computing device 200.

The computing device 200 may be, but is not limited to, a smartphone, amobile device, a tablet computer, a notebook computer, a desktopcomputer or a work station. In a preferred embodiment, the computingdevice 200 is a smartphone capable of receiving Bluetooth signals fromthe multi-mic sound collector 100. It is to be understood that datatransmission and communication among the multi-mic sound collector 100,the computing device 200, and the server 300 may be performed via USB,micro USB, serial port, IEEE1394, Bluetooth, Wi-Fi, Infrared, ZigBee,WiMAX, 3G, 4G, 4G LTE, 5G, or any other commonly known wired or wirelesstransmission means.

Referring now to FIG. 6. The processor 230 of the computing device 200includes a feature extractor 410, a classifier 420, and a signallocalizer 430. The feature extraction module 410 is configured toidentify and extract preliminary audio features from the synchronizedaudio data received from the synchronizer 220. The classifier 420 isconfigured to separate and classify the extracted preliminary audiofeatures received from the feature extractor 410 to single out a targetaudio feature. The signal localizer 430 is configured to analyze thetarget audio feature obtained by the classifier 420 by performingdirection of arrival (DOA) estimations, such as multiple signalclassification (MUSIC), beamforming and other data processingalgorithms, to obtain locational information of the source of the targetaudio feature. The MUSIC algorithm may include the group delay MUSIC(GD-MUSIC), recursively applied and projected MUSIC, minimum variancedistortionless response (MVDR), and linear constraint minimum variance(LCMV) algorithms. The beamforming algorithm may include the generalizedsidelobe canceller (GSC) algorithm. The processor 230 may be an IntelCore i7 processor, ARM processor, x8086 or other processor orapplication specific integrated circuit having similar computingcapability.

The processor 230 may further include a data analyzer (not shown infigure) configured to compare the obtained target audio feature andaudio source location with previous audio record of the subject 10,audio data of healthy or normal subjects, or default settings stored inthe memory 250 to obtain a diagnostic result. For example, mid-to-latesystolic heart murmur detected at the upper left part of the heart couldindicate mitral valve prolapse (MVP).

Referring now to FIG. 7. According to various embodiments of the presentinvention, sound localization method of the sound localization systemincludes the steps of: (S1) acquiring sound data of a subject; (S2)identifying a target audio feature from the sound data; and (S3)analyzing the target audio feature to obtain the locational informationof the source of the target audio feature.

Acquisition of sound data as in Step S1 is performed by the multi-micsound collector 100 of the sound localization system of theaforementioned embodiments to collect sounds coming from a subject orobject or from a defined space.

In an embodiment, identification of a target audio feature from thesound data as in Step S2 includes the steps of: (S21) preprocessing thesound data; (S22) extracting preliminary audio features; and (S23)separating and classifying the preliminary audio features to obtain thetarget audio feature. Preprocessing of the sound data as in Step S21includes the steps of: (S211) converting and encoding the acquired sounddata into digital audio data; (S212) preprocessing the digital audiodata; and (S213) synchronizing the audio data. Processing of the digitalaudio data as in Step S212 includes as down sampling, positioning,tagging, augmenting, filtering, and time stamping of the digital audiodata.

In an embodiment, extraction of preliminary audio features as in StepS22 is performed by processing the synchronized audio data according tothe voice activity detector (VAD), Mel-frequency cepstral coefficient(MFCC), and K-means algorithms. In an embodiment, separation andclassification of the extracted preliminary audio features as in StepS23 may include the steps of: (S231) separating noise from thepreliminary audio features; (S232) classifying normal and abnormal audiofeatures; and (S233) separating undesired abnormal audio features toobtain a target audio feature; and are performed according to theK-nearest neighbor (KNN), Gaussian mixture model (GMM), support vectormachine (SVM), or deep neural network (DNN) algorithm. In an exemplaryembodiment where the sound receivers 110 of the multi-mic soundcollector 100 are stethoscopic chestpieces that are optimized fordetecting heart sounds, more than one abnormal audio features may bedetected from the acquired heart sounds. When the 4^(th) heart sound isthe target sound feature (or the sound of interest), Step S233 would beperformed to eliminate abnormal audio features that are not associatedwith the 4^(th) heart sound, thereby singling out the target audiofeature (eg. the 4^(th) heart sound as in this example) for furtheranalysis.

The step of analyzing the target audio feature as in Step S3 includesthe step of: (S31) defining coordinates of the sound receivers 110; and(S32) performing direction of arrival (DOA) estimations on the targetaudio feature to obtain the locational information of the source of thetarget audio feature. The DOA estimation may include multiple signalclassification (MUSIC), such as GD-MUSIC, recursively applied andprojected MUSIC, MVDR, and LCMV algorithms, beamforming, such as GSCalgorithm, and other data processing algorithms. Step S3 may furtherinclude the step of: (S33) comparing the target audio feature and thelocational information of the source of the target audio feature withstored data to obtain a diagnostic result. The stored data may beprevious audio record of the subject, audio data of healthy or normalsubjects, or default settings stored in the system.

The sound localization method may further include the step of: (S4)visualizing the locational information of the source of the target audiofeature. The locational information may be visualized on a coordinatesystem defined by positions of the sound receivers 110 or over an uni-or multi-dimensional virtual image or model of the subject so as toprovide a more intuitive diagnostic result. The model may be ananatomical model reconstructed from a photo image, an X-ray image,computed tomographic image, or magnetic resonance image. Alternatively,the anatomical model may be a template model established from ananatomical atlas of the body, such as the chest region, head and neckregion, and abdominal region.

According to some exemplary embodiments, the location of an abnormalsound may be visualized one-dimensionally between the right chest up(RCU) and left chest up (LCU) spots, as illustrated in FIG. 8A, ortwo-dimensionally on a plane defined by the right chest down (RCD), leftchest down (LCD), RCU, and LCU spots, as illustrated in FIG. 8B, or morepreferably three-dimensionally in a space defined by the anterior chestthymus (FCT), anterior chest (FC) and posterior chest (BC) spots, asillustrated in FIG. 8C.

As exemplified in FIGS. 9A and 9B, the diagnostic result may also bevisualized by an application on the computing device 200. For example,as illustrated in FIG. 9A, the application may show the recordedsoundwaves and sound intensity and frequency, illustrate the locationsof the sound receivers, and indicate the diagnostic result (such as thephysiological condition of the subject). When abnormal sound occurs, asillustrated in FIG. 9B, the application may present an alert message andindicate the locations at which the abnormal sound is detected.

The sound localization method may further include the step of: (S5)recording and storing data associated the target audio feature. The datamay include type, frequency, intensity and location of the audiofeature, as well as chronological changes in frequency, intensity, andlocation of the audio feature.

Referring now to FIG. 10. The sound localization system according to anexemplary embodiment of the present invention may be applied tomonitoring of breathing sound and detection asthmatic symptoms. In theexample, sound receivers of the sound collector are stethoscopicchestpieces integrated in the form of patches, which are removablyattached to five selected spots of a patient (S91); as illustrated inFIG. 3, the spots may be the right and left upper lobes of the lungregion at anterior chest, right and left lower lobes of the lung regionat posterior chest, and the bronchial region. After the sound receivingpatches are attached to the patient, the patches are connected to apreprocessing device (S92); the preprocessing device includes a MCU thatis configured to initiate a sound collection (or auscultation) routineby the patches periodically for a default duration of time. Thepreprocessing device is switched on and coupled to a smartphone viaBluetooth (S93). The user may set from an application installed on thesmartphone the time, duration and items for monitoring, as well as thetype of alert sent in case of detection of abnormal events, and startthe monitoring program (S94).

Once started, the MCU initiates a sound collection routine (eg.sequential auscultation by the five sound receiving patches) every tenminutes and wirelessly transmits the audio data to the smartphone foralgorithmic analyses of the data to identify a target audio feature,such as wheezing, or other pathological sound features (S95). When noabnormality is detected, the MCU would continue the auscultation routineuntil the set duration of monitoring ends (S96); thereafter, all of thecollected audio data is wirelessly transmitted to a cloud server forstorage and trend analysis (S97); further, the monitoring record andanalysis results in the server may be transmitted back to the smartphonefor detailed diagnosis and follow-ups by the user or physicians.Alternatively, when wheezing is detected, the smartphone applicationwould issue an alert ringtone and/or message and indicate treatmentadvices (S98); meanwhile, the MCU stops the periodic auscultationroutine and performs continuous auscultation until the alert iscanceled.

Likewise, as illustrated in to FIG. 11. The sound localization systemaccording to an exemplary embodiment of the present invention may beapplied to monitoring of heart sound and detection arrhythmic symptoms.In the example, sound collectors are in the form of patches, each ofwhich includes a stethoscopic chestpiece and a MCU, as illustrated inFIG. 1. Two sound collecting patches are removably attached to the heartregion at front and posterior chest of a user (S101). The MCU isconfigured to initiate recording of heart sound and heart rate by thestethoscopic chestpiece. After the sound collecting patches are attachedto the user, the patches are switched on and coupled to a smartphone viaBluetooth (S102). The user may confirm from the application installed onthe smartphone status of connection between the patches and thesmartphone and check audio features of the detected heart sound, heartrate, and calculated heart rate variability under an active mode. Theuser may also switch the system to a standby mode from the application;the MCUs of the sound collecting patches would stop heart sound andheart rate recording under the standby mode.

When experiencing irregular heartbeat or chest pain, the user may switchthe system back to the active mode from the application, and activaterecording of heart sound and heart rate by the sound collecting patches(S103). When abnormal heart sound or heart rate is not detected (S104),the sound collecting patches would start a auscultation routine (eg.continuous sound collection and recording for three minutes) and returnto the standby mode (S105). Alternatively, when abnormal heart sound orheart rate is detected (S104), the smartphone application would issue analert ringtone and/or message and indicate treatment advices to the user(S106); the smartphone may also send a notification of such event to adesignated hospital or medical institution for necessary emergencyresponses. Meanwhile, the sound collecting patches would repeat theauscultation routine (S105) and return to the standby mode only untilthe heart rate recovers. All of the collected auscultation data iswirelessly transmitted to a cloud server for storage and trend analysis(S107); the monitoring record and analysis results in the server may betransmitted back to the smartphone for detailed diagnosis and follow-upsby the user or physicians.

In sum, the present invention according to the preferred embodimentscouples a multi-mic sound collector with sound analysis and spatialanalysis to identify abnormal or pathological sounds coming from amonitored subject and obtain detailed locational information of thesource of the abnormal sounds. The present invention facilitatesmonitoring of the functioning or physiological signs of the subject andallows early detection and diagnosis of abnormalities or diseases.

Previous descriptions are only embodiments of the present disclosure andare not intended to limit the scope of the present disclosure. Manyvariations and modifications according to the claims and specificationof the disclosure are still within the scope of the claimed disclosure.In addition, each of the embodiments and claims does not have to achieveall the advantages or characteristics disclosed. Moreover, the abstractand the title only serve to facilitate searching patent documents andare not intended in any way to limit the scope of the claimeddisclosure.

What is claimed is:
 1. A sound localization system, comprising: amulti-mic sound collector comprising a plurality of sound receivers forcollecting sound data of a subject; and a computing device incommunication with the multi-mic sound collector, comprising: a datacommunicator for receiving preprocessed audio data from the multi-micsound collector; a synchronizer electrically connected to the datacommunicator for synchronizing the preprocessed audio data; and aprocessor electrically connected to the synchronizer for analyzing thesynchronized audio data to identify and localize a target audio feature.2. The sound localization system according to claim 1, wherein themulti-mic sound collector comprises: a carrier; and a plurality of soundcollecting modules removably attached to the carrier, each of the soundcollecting modules comprising: one of the plurality of sound receivers;a convertor electrically connected to the sound receiver for convertingthe sound data into digital audio data; and a microcontroller (MCU)electrically connected to the convertor for controlling sound collectionby the sound receiver and preprocessing the digital audio data.
 3. Thesound localization system according to claim 1, wherein the multi-micsound collector comprises: a carrier; the plurality of sound receiversremovably attached to the carrier; and a preprocessing device,comprising: a convertor electrically connected to the sound receiversfor converting the sound data into digital audio data; and amicrocontroller (MCU) electrically connected to the convertor forcontrolling sound collection by the sound receivers and preprocessingthe digital audio data.
 4. The sound localization system according toclaim 1, wherein the processor of the computing device comprises: afeature extractor for identifying and extracting preliminary audiofeatures from the synchronized audio data; a classifier for separatingand classifying the preliminary audio features to obtain the targetaudio feature; and a signal localizer for analyzing the target audiofeature to obtain locational information of a source of the target audiofeature.
 5. The sound localization system according to claim 4, whereinthe processor further comprises: a data analyzer for comparing theobtained target audio feature and the location of the source of thetarget audio feature with data stored in the computing device to obtaina diagnostic result.
 6. The sound localization system according to claim1, wherein the computing device is further in communication with aserver for data analysis and storage.
 7. The sound localization systemaccording to claim 1, wherein the sound receivers are a plurality ofstethoscopic chestpieces for auscultating the subject.
 8. The soundlocalization system according to claim 1, wherein an amount of the soundreceivers are at least three.
 9. The sound localization system accordingto claim 1, wherein at least a portion of the sound receivers arearranged rectangularly over a chest of the subject.
 10. The soundlocalization system according to claim 1, wherein at least a portion ofthe sound receivers are arranged rectangularly over a heart of thesubject.
 11. The sound localization system according to claim 1, whereinat least a portion of the sound receivers are arranged triangularly overa heart of the subject.
 12. The sound localization system according toclaim 1, wherein at least a portion of the sound receivers are arrangedat left and right costophrenic angles at a posterior chest of thesubject.
 13. A sound localization method for a sound localization systemcomprising a plurality of sound receivers for collecting sound data of asubject and a computing device in communication with the soundreceivers, comprising steps of: acquiring sound data of the subject;identifying a target audio feature from the sound data; and analyzingthe target audio feature to obtain locational information of a source ofthe target audio feature.
 14. The sound localization method according toclaim 13, the step of identifying a target audio feature from the sounddata comprising steps of: preprocessing the sound data; extractingpreliminary audio features; and separating and classifying thepreliminary audio features to obtain the target audio feature.
 15. Thesound localization method according to claim 14, the step of extractingpreliminary audio features is performed according to a voice activitydetector (VAD) algorithm, a Mel-frequency cepstral coefficient (MFCC)algorithm, and a K-means algorithm.
 16. The sound localization methodaccording to claim 14, the step of separating and classifying theextracted preliminary audio features comprising steps of: separatingnoise from the preliminary audio features; classifying normal andabnormal audio features; and separating undesired abnormal audiofeatures to obtain the target audio feature.
 17. The sound localizationmethod according to claim 14, the step of separating and classifying theextracted preliminary audio features is performed according to aK-nearest neighbor (KNN) algorithm, a Gaussian mixture model (GMM)algorithm, a support vector machine (SVM) algorithm, or a deep neuralnetwork (DNN) algorithm.
 18. The sound localization method according toclaim 13, the step of analyzing the target audio feature comprising astep of: performing direction of arrival (DOA) estimations on the targetaudio feature to obtain the locational information of the source of thetarget audio feature.
 19. The sound localization method according toclaim 13, further comprising a step of: comparing the target audiofeature and the locational information of the source of the target audiofeature with stored data to obtain a diagnostic result.
 20. The soundlocalization method according to claim 13, further comprising a step of:visualizing the locational information of the source of the target audiofeature over a multi-dimensional image of the subject.